Gaze Awareness for Video-Conferencing: A Software Approach

bringing gaze awareness to desktop videoconferencing have relied on hardware solutions. Here, the authors describe their software approach, which tracks participants’ head and eye movements using vision techniques, then uses this information to graphically place the head and eyes in a 3D environment. T he impact of gaze is striking. We all know the experience of “feeling watched,” and it’s hard to resist the urge to look at someone who’s staring at you. As the sidebar, “Human Gaze: A Closer Look,” explains, gaze awareness—and eye contact in particular—is extremely important in face-to-face communication. In addition to practical uses, such as using gaze to direct turn-taking in conversation, gaze awareness has more abstract social value: People who use frequent eye contact are perceived as more attentive, friendly, cooperative, confident, mature, and sincere than those who avoid it. Despite the obvious importance of directed gaze, most videoconferencing systems make it impossible for participants to make eye contact or even to determine where or at what the other participants are looking. This loss of gaze awareness has a profound impact on communication, and may in fact be a contributing factor in the failure of videoconferencing to meet with its long-anticipated success (for more on this, see the sidebar, “Videoconferencing Research” on page 28). The lack of gaze awareness in typical videoconferencing systems stems from the fact that when participants look at each other, they stare into their displays rather than into the camera, which is typically mounted above, below, or beside the display. Unless people look directly into the camera, you will never perceive them as making eye contact with you, no matter where you’re situated in relation to the display. Conversely, if they’re looking into the camera, they will always appear to be looking at you, even when you move around. A famous example of this is Leonardo da Vinci’s painting of the Mona Lisa, whose eyes appear to follow viewers around the room. As Figure 1 shows, videoconferencing typically gives each participant an individual, arbitrarily placed window, and participants never appear to look at each other. In addition to inhibiting eye contact between two parties, with multiparty desktop systems participants have no sense of spatial relationship and cannot tell who’s looking at whom. Existing systems restore gaze awareness only by using special and typically expensive hardware. We developed a software-based approach to videoconferencing gaze awareness. We aim to develop an inexpensive videoconferencing system that supports two to four participants. To date, hardware has been a serious obstacle to videoconferencing. Thus, it was important that our system work with cheap, commonly available hardware. The system uses standard PC hardware, equipped with audio I/O and video capture. Virtually all PCs now ship with sufficient audio support, and many are beginning to ship with video capture. For those